The practical utility of incorporating model selection uncertainty into prognostic models for survival data

Predictions of disease outcome in prognostic factor models are usually based on one selected model. However, often several models fit the data equally well, but these models might differ substantially in terms of included explanatory variables and might lead to different predictions for individual patients. For survival data we discuss two approaches for accounting for model selection uncertainty in two data examples with the main emphasis on variable selection in a proportional hazard Cox model. The main aim of our investigation is to establish in which ways either of the two approaches are useful in such prognostic models. The first approach is Bayesian model averaging (BMA) adapted for the proportional hazard model (Volinsky et al., 1997). As a new approach we propose a method which averages over a set of possible models using weights estimated from bootstrap resampling as proposed by Buckland et al. (1997), but in addition we perform an initial screening of variables based on the inclusion frequency of each variable to reduce the set of variables and corresponding models. The main objective of prognostic models is prediction, but the interpretation of single effects is also important and models should be general enough to ensure transportability to other clinical centres. In the data examples we compare predictions of the two approaches with “conventional” predictions from one selected model and with predictions from the full model. Confidence intervals are compared in one example. Comparisons are based on the partial predictive score and the Brier score. We conclude that the two model averaging methods yield similar results and are especially useful when there is a high number of potential prognostic factors, most likely some of them without influence in a multivariable context. The new method based on bootstrap resampling has the additional positive effects of achieving model parsimony by reducing the number of explanatory variables and dealing with correlated variables in an automatic fashion. keywords model selection uncertainty; survival analysis; model averaging; bootstrap; prognostic factor models.